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METHOD AND SYSTEM FOR TRANSMITTING MEDIA STREAMS OVER A 

VARIABLE BANDWIDTH NETWORK 

FIELD OF THE INVENTION 

The present invention relates to methods and systems for transmitting 
media streams in general, and to methods and systems for transmitting media 
streams over variable bandwidth communication lines, in particular. 

BACKGROUND OF THE INVENTION 
5 l\/lethods and systems for transmitting media streams over variable 

bandwidth channels, are known in the art. 

US Patent No. 5.847.760 Issued to Elmaliach et al. and entitled 
"Method for Managing Video Broadcast", is directed to a method for broadcasting 
compressed digital video signals. An encoder encodes each video frame 

10 according to the maximal bit rate, the half-maximal bit rate, and the quarter of the 
maximal bit rate. Each frame contains a plurality of a group of pictures (GOP). 
Each frame contains the same number of GOP's. and the GOP's are placed in 
the same order in all three types of frames. A transmitter detects the available 
bandwidth in the communication link, and provides this information to a controller. 

15 The controller selects one of the three types of encoded frames. If the available 
bandwidth is maximal, the controller selects the maximal bit rate frame type. If the 
available bandwidth is less than the maximal but more than half, the controller 
selects the half-maximal frame type, and if the availablis bandwidth is less; than 
half the maximal, then the controller selects the quarter bit rate frame. A receiver 

20 receives the compressed frame from the transmitter and forwards it to a decoder, 
which decodes the frame and displays it on a display. 

Methods for multiplexing a plurality of media streams into a single 
channel, where each media stream Is assigned a different bandwidth, are also 
known in the art. One such method analyzes the activity in the streams and 

25 provides feedback to the encoders to encode according to a selected bit rate. 
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The Rate-Mux 300, Cisco V-blts Corp. Implements another method In 
which the unit's programmable statistical re-multiplexing engine performs real- 
time CBR (Constant Bit Rate) and VBR (Variable Bit Rate) rate conversion and 
bit-stream manipulation processing to the input streams. Using a statistical 

5 model, the processed streams are then recombined into a highly efficient, 
multiplexed program transport stream for retransmission. 

US Patent no. 5966120 to Arazi et al, is directed to "Method and 
apparatus for combining and distributing data with pre-formatted real-time video". 
This reference describes a system, which transmits a plurality of streams via a 

10 constant bit rate channel. Each of the streams can be of a variable bit rate 
channel but the system assures that the total sum of all if the streams at any 
given point in time does not exceed the channel bandwidth, even if from time to 
time, not all of it is fully used. The system described in this reference attempts to 
make use of excess bandwidth, which is variably available and insert other types 

15 of data therein, such as commercials. This system requires a client unit at the 
receiving end to extract the transmitted streams and the embedded additional 
data. 

US Patent No. 5,629,736 issued to Haskell et al. and entitled "Coded 
Domain Picture Composition for Multimedia Communications Systems", is 

20 directed to a method for composition of video signals. Up to four video signals 
from four sources are fed to the system. The bit rates of the four signals may be 
different. Each video signal is decoded and processed by a discrete cosine 
transformation processor. The four discrete cosine transformation processors are 
controlled by a controller, whereby the bit rates of all four signals are converted to 

25 a single bit rate. The four video signals of equal bit rates are then synchronized 
and multiplexed, and fed to a display unit as one video signal, to display the four 
video images simultaneously. 

US Patent No. 5,216,503 issued to Paik. et al. and entitled "Statistical 
Multiplexer for a Multichannel Image Compression System", Is directed to a 

30 system for compression of a plurality of video signals and decompression thereof. 
Each of a plurality of video signals is encoded by an encoder. The encoded video 
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signals are multiplexed and transmitted as a single video signal to a receiver. The 
video signal is then demultiplexed and decoded by a number of decoders, each 
corresponding to an encoder at the transmitting end. Each decoder then outputs 
the original video signal encoded at the transmission end. 
5 MPEG standard provides a dual layer method of encoding, which Is 

called SNR scalability. According to this method, a stream is encoded into two 
encoded streams, a basic level stream and a supplemental enhanced level 
stream. The layering is performed once for the entire movie and not on a block 
level. Either the base layer or both layers can be used to produce the original 
10 stream. The layer selection decision has to be performed prior to transmission. 
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SUMMARY OF THE PRESENT INVENTION 

It is an object of the present invention to provide a novel method and 
system for scalable transmission of media streams, which overcomes the 
disadvantages of the prior art. 
5 In accordance with a preferred embodiment of the present Invention, 

there is provided a method for providing scalable bandwidth transmission of a 
media stream. The method includes the steps of: detemiining a plurality of 
compression levels for the media stream, and defining the highest level of 
compression of the compression levels as a base level. The can further include 
10 the steps of compressing the media stream according to the compression levels, 
thereby producing a plurality of compressed versions of the media stream, and 
defining a selected one of the compressed versions a base layer. The selected 
compressed version is compressed according to the base level. The method can 
further include the step of producing a supplemental layer for each successive 
15 pair of the compressied versions. 

The method can also include the step of providing the base layeir as a 
reconstructed media stream or producing a reconstructed media stream from the 
base layer and at least a portion of at least one of the supplemental layers. 

According to the available bandwidth, base layer and successive 
20 supplemental layers, can be transmitted to a remote location. Alternatively, a 
reconstructed media stream can be transmitted to a remote location, either in an 
encoded or a decoded format. 

The of reconstructed media stream can be packetized before 
transmitting to a remote location or intermediate storage thereof 
25 It is noted that the media stream can be any compressible stream such 

as an MPEG encoded media stream, an original media stream, a JPEG media 
stream, other types of elementary streams Video stream such as an audio 
stream, a data stream, complexed streams which have a plurality of other 
streams embedded therein, and the like 
30 In accordance with another aspect of the present invention, there is 

provided a method for reconstructing a portion of a media stream from a portion 
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of a base layer stream and a respective portion of at least one supplemental layer 
stream including the step of adding the respective portion of at least one 
supplemental layer stream to the base layer stream. 

In accordance with another aspect of the present invention, there is 
provided a method for providing scalable bandwidth transmission of a plurality of 
media streams. The method includes the steps of determining a group of 
compression levels for each of the media streams, and defining the highest level 
of compression of the compression levels of each the media streams, as a base 
level thereof. 

The method can further include the steps of: compressing each of the 
media streams according to a respective one of the groups of the compression 
levels, thereby producing a group of compressed versions, for each of the media 
streams, and defining a selected one of the compressed versions of a selected 
group of compressed versions, a base layer. The selected compressed version is 
compressed according to the base level, of the selected group of compression 
levels. 

The method can further include the step of producing a supplemental 
layer for each successive pair of the compressed versions of a selected group, 
thereby defining a group of supplemental layers. The method can further include 
the steps of associating between portions of different ones of the base layers, 
and associating between portions of different ones of the supplemental layers of 
the same order, of different ones of the groups. For example, associating 
between a portion of the second layer of a first media stream and a respective 
portion of the second layer of a second media stream. 

The portions of the associated base layers can then be provided as 
reconstructed media streams. Alternatively, the method can include a step of 
producing reconstructed media streams, each from a respective one of the 
portions of the associated base layers and from at least one of a respective one 
of the portions of the supplemental layers. It is noted that the decision of which is 
the highest level used in the reconstruction process can be made according to a 
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plurallty of considerations such as available bandwidth, received packets at each 
layer, etc. 

The associated portions of the base layers and supplemental layers 
can be transmitted to a remote location. Alternatively, the method can further 

5 include the step of transmitting the reconstructed media streams to a remote 
location. An optional step of the invention includes packetizing the reconstructed 
media streams, (for example, according to MPEG Transport) which then can be 
transmitted packetizes. 

In accordance with another preferred embodiment of the present 

10 invention, there is provided a method for reconstructing portions of media 
streams from portions of a base layer streams and a respective portions of at 
least one supplemental layer streams. The method includes the steps of: 
detecting an association between each the base layer streams and at least a 
respective portion of at least one respective supplemental layer streams, and 

15 adding each of the base layer streams and the associated respective portion of at 
least one respective supplemental layer. 

reconstrute_stream= base^layer+ ^supplemerttLlayef 

»=0.I.2...k 

In accordance with another preferred embodiment of the present 
invention there is provided a scalable bandwidth transmission media stream 

20 generating apparatus. The apparatus includes a media analyzer and a layering 
generator connected to the media analyzer. The media analyzer receives a 
media strearn and determines a layering structure therefore. The layering 
generator processes the media stream according to the layering structure, 
thereby producing a base layer and a plurality of supplemental layers. 

25 The apparatus can further include a storage unit, connected to the 

layering generator, wherein the storage unit is divided to a plurality of queues and 
each the queues stores a selected one of the base layers and the supplemental 
layers. 

The apparatus can further include a routing controller, connected to the 
30 storage unit, for retrieving selected successive ones of the base layers and the 
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supplemental layers. In addition, the apparatus of the Invention can also Include a 
layer assembler, connected to the routing controller, for producing a 
reconstructed media stream from the base layer and at least a successive one of 
the supplemental layers, received from the routing unit. 
5 In accordance with a further preferred embodiment of the present 

invention, there is thus provided a scalable bandwidth transmission media stream 
generating apparatus. The apparatus includes a plurality of media analyzers, and 
a plurality of layering generators, each associated with and connected to a 
selected one of the media analyzers. Each of the media analyzers receives a 

10 different media stream and determines a layering structure therefore. Each of the 
layering generators processes the media stream according to the layering 
structure, thereby producing a base layer and a plurality of supplemental layers. 

The apparatus can further include a storage unit, connected to the 
layering generator. The storage unit is divided to a plurality of queues, each 

15 storing a selected one of the base layers and the supplemental layers. For 
example. queue(O) stores the base layers of all of the streams, queue{1) stores 
the first supplemental layel- of all of the streams, queue(2) stores the second 
supplemental layer of all of the streams, whose layering structure includes at 
least two supplemental layer and so on and so forth. 

20 The apparatus can also include a routing controller, connected to the 

storage unit, for retrieving selected successive ones of the base layers and the 
supplemental layers, as well as a layer assembler an-ay. connected to the routing 
controller, for producing a reconstructed media streams from the base layers and 
at least a successive ones of the supplemental layers, received from the routing 

25 unit. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention will be understood and appreciated more fully 
from the following detailed description taken in conjunction with the drawings in 
which: 

30 Figure 1 is a visual representation of block OCT values, at a 

predetermined quantizing value; 
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Figures 2A, 2B, 2C, 2D and 2E are visual representations of the block 
of Figure 1, at different quantizing values, In accordance with a preferred 
embodiment of the present Invention; 

Figures 3A, 3B, 3C, 3D and 3E are representation of a layer structure, 
in accordance with another preferred embodiment of the present invention; 

Figure 4 is a representation of a packetizatlon structure, in accordance 
with a further preferred embodiment of the present Invention; 

Figure 5 Is a schematic illustration of an assembly mechanism 
structure, constructed and operative In accordance with another preferred 
embodiment of the present invention; 

Figure 6 is a schematic illustration of an apparatus, constructed and 
operative in accordance with another preferred embodiment of the present 
Invention; 

Figure 7 is an illustration of a method for operating the transmission 
system of the apparatus of Figure 6, operative in accordance with another 
preferred embodiment of the present invention; 

Figure 8 is an illustration of a method for operating the reception- 
system of the apparatus of Figure 6, operative in accordance with another 
preferred embodiment of the present invention; 

Figures 9A, 9B, 9C and 9D, each provide a schematic illustration of the 
layering distribution of a different media stream on the same set of layers, at 
different quantizing values; 

Figure 10, which is a schematic illustration of an apparatus, down 
streaming media through a plurality of communication channels, constructed and 
operative in accordance with another preferred embodiment of the present 
Invention; and 

Figure 1 1 Is a schematic illustration of a video re-multipiexing system, 
constructed and operative in accordance with another preferred embodiment of 
the present invention. 
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DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

The present Invention overcomes the disadvantages of the prior art by 
providing a novel method and system for separating a media stream into layers, 
each defining a different quality, each added onto a lower quality one, for 
5 transmitting the layers over a variable bandwidth communication line and for 
assembling the layers Into a media stream at the receiving end. 

Although the present invention Is described predominantly in terms of 
the transmission and storage of video and audio infomiation encoded in 
accordance with the MPEG format, the concepts and methods are broad enough 

10 to encompass distribution systems using other data compression techniques and 
other data formats. Throughout this detailed description, numerous details are 
specified such as program types and data stream structures, in order to provide a 
thorough understanding of the present invention. 

Both MPEG-1 (ISO/IEC 11172) and MPEG-2(ISO/IEC 13818) have a 

15 few chapters. The following description mainly addresses two of them. Chapter 1 
(11172-1, 13818-1) is directed to Systems. In MPEG-2 this includes Transport 
(herein after referred to as Systems or Transport, which is a specific System 
type). Chapter 2 (11172-2, 13818-2) is directed to video compression (herein 
after referred to as MPEG-Video). 

20 MPEG-Video sets rules on how to encode video and what Is required in 

order to have a legal MPEG-Video stream. It requires buffer handling. An MPEG- 
Video stream Is called an elementary stream. Other elementary streams can be 
audio streams (Dolby AC-3, MPEG-Audio or anything else), data streams and 
any other stream. 

-5 The elementary streams can include any so-called private streams. A 

receiver can do whatever it desires with what it receives, providing that It includes 
the means to extract these private streams from the received data. Usually this 
would require some kind of client installed on the receiver In order to handle the 
private data as the transmitter intended. 

10 MPEG-Systems also requires buffer handling. In principal, each 

elementary stream has its own buffer handling and the overall multiplexing, which 
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is defined in MPEG-Systems, has its own buffer handling. A private stream has 
no specified buffer handling and it is up to the transmitter and receiver to process 
it according to their predetermined purpose. 

A program includes one or more elementary streams. For example, the 

5 program can include one or more video streams (for example, In a multl angle 
case, where each video stream provides s different angle of the same clip), one 
or more audio streams and the like. MPEG Systems the provisions for defining 
which elementary streams define a program, and different timing instructions so 
that the streams will be synchronized. It is Important to note that all timing 

10 considerations are done at the MPEG-Systems level. In addition, MPEG- 
Transport allows the transmission of one or more programs - all multiplexed on 
one stream. The standards provide the option to state which streams belong to 
which program. This is facilitated by the fact that every transmitted elementary 
stream has a unique identification (ID) number called PID (Packetized ID). 

15 MPEG-Video can be provided at a constant bit-rate (CBR) or at a 

variable bit-rate (VBR). MPEG Transport can technically also be one of either but 
conventionally only CBR transport Is used. Usually, each program may be VBR 
but their sum is CBR at the Transport's total bit rate. 

The present invention provides a novel method, which multiplexes a 

20 plurality of VBR programs in such a way, that their sum never exceeds the total 
Transport bit rate and that no bandwidth be wasted. This is done by analyzing the 
encoded streams, altering them, producing new legal encoded streams and 
multiplexing the new streams into Transport format so that the total isn't 
exceeded and all the bandwidth is used and no client is required at the receiving 

25 end. 

As stated above, the examples set forth in the following description are 
provided with respect to MPG ISO/IEC 11172-2 and ISO/IEC 13818-2 video 
compression standards. A picture, which is to be encoded includes either a 
complete frame, or a field, which is the even or odd lines of a complete frame. 
30 The above standards define three types of encoded pictures, an l-frame, a B- 
frame and a P-frame. An l-frame (Intra-frame) includes the entire information 
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which is required to reconstruct the encoded picture. A P-frame (Predictive) 
includes information, which, in combination with previously encoded P-frames 
and l-frames, can reconstruct the encoded picture. A B-frame (Bi-directional) 
includes information, which, in combination with previously encoded l-frames and 

5 P-frames. can reconstruct the encoded picture. 

A picture to be encoded is divided Into components at different levels. A 
slice includes a predetermined number of lines (for example 16). A block includes 
a matrix of 8X8 pixels. A macro-block includes a 2X2 matrix of luminance blocks 
(which results in a 16X16 matrix of pixels). 

10 Each block of pixels is transformed using Discrete Cosine Transform 

(DCT) which produces a respective sequence of values. These values are then 
divided by a quantizing parameter, also called the quantizing scale and only the 
integer portion of the results, is kept. For example, If the DCT results in the 
following sequence 16, 5, 10, 2, 0, 4 and the quantizing scale is 6, then the 

15 resulting sequence is 2, 0, 1, 0, 0, 0. It is noted that a different quantizing value 
can be used for every macro-block. 

After the DCT and quantizing stages, the set of values, preferably 
Includes a large number of zero values. The values are further encoded using a 
method called Run Level Encoding (RLE), which transforms the sequence Into 

20 pairs of number, a value (greater than zero) and the number of zero values that 
preceded it. 

The present Invention provides a method in which a quantized 
sequence is further quantized to a plurality of higher quantization levels (higher 
compression, lower quality). The highest quantization level Is set to be the base 
25 level. The output data includes the base level and the difference between each 
two adjacent levels, up to the original one. 

The following description addresses a stream component, which is a 
single frame. It is noted, for the purpose of the present Invention, that frame can 
be replaced by any other stream component such as a slice, a macro-block, a 
30 block, a field and the like. 



BNSOOCIO: <WO_0072601ASLl_> 



wo 00/72601 



PCT/USOO/14716 



-12- 

Reference is now made to Figure 1 , which is a visual representation of 
block DCT values, at a predetermined quantizing value, after basic quantizing. As 
seen in Figure 1 , most of the values are greater than zero. Reference is now 
made to Figures 2A. 2B, 2C, 2D and 2E. which are visual representations of the 
block of Figure 1 , at different quantizing values, in accordance with a preferred 
embodiment of the present invention. Figure 2A, 2B. 2C. 2D and 2E represent 
the block values at respective quantizing values of Q=5, 4, 3, 2 and 1 (e.g.. 
Figure 2A represents the integer portions of the results of a division of the values 

of Figure 2E, by 5. 

Reference is now made to Figures 3A, SB, 3C, 3D and 3E. which are 
representation of a layer structure, in accordance with another preferred 
embodiment of the present invention. Figure 3A is a visual representation of the 
block of Figure 1 , at the highest quantizing value and is Identical to Figure 2A. 
Figure 3B represents the difference (A) between the representations of Figures 
2A and 2B, so that adding the representation of Figure 2B over the one of Figure 
3A, yields in the representation of Figure 2B. similarly. Figure 3C represents the 
difference (A) between the representations of Figures 2B and 2C, Figure 3D 
represents the difference (A) between the representations of Figures 2C and 2D 
and Figure 3E represents the difference (A) between the representations of 
Figures 2D and 2E. 

The newly compressed frames at each of the compression levels 
(Figures 2A, 2B, 2C, 2D and 2E) can now be used to reproduce a representation 
of the media stream (deteriorated according to the compression level) or be sent 
to a far end of a communication network. Alternatively, the base layer (Figure 3A) 
and as many successive supplemental layers (Figures 3B, 3C, 3D and 3E), can 
now be used to reproduce the media stream or be sent to a far end of a 
communication network. The present Invention provides a further packetlzing 
stage, which can be also be used for re-multiplexing of a plurality of media 
streams, as will be described further below. 

Reference Is further made to Figure 4, which is a representation of a 
layering and packetlzing mechanism structure, constructed and operative in 
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accordance with a further preferred embodiment of the present invention. The 
structure includes a layering generator 70, a packetizing unit 72, a plurality of 
queues, generally referenced 60 and a transmission router 74. 

Each frame yields a plurality of firame representations, generally 
referenced SOg. where i denotes a frame index and j denotes a layer index. The 
first frame representations are provided at all of the compression layers from the 
base level, at the highest quantization level, to the rest, which are consecutive A 
portions for lower quantization levels. For example, frame representation 50o.o. is 
the representation of the first frame at the base layer (highest compression - 
lowest quality), frame representation 50o.i. is the representation of the first frame 
at the second layer level (lower compression - higher quality). Frame 
representation 50o.n. is the representation of the first frame at the last layer 
(lowest compression - highest quality). The same applies for frame M. references 
50m.o and 50m.n 

The frame representations SOg are arranged in such a structure, so that 
each can be accessed individually, thereby enabling reconstruction of the original 
stream in a variety of ways, at different qualities, which can vary from frame to 
frame. 

Packetizing unit 72, encapsulates the frame representations so that the 
original stream can be constructed from only some of them, where some packets 
are discarded. This aspect of the invention provides robustness, in situations 
where bandwidth decreases, and not all of the frame representations of a 
selected picture arrive in due time at the receiving end. 

The packetizing unit 72 packetizes the frame representation, respective 
of a selected layer and provides the packets to respective layer queues 60o. 60i. 
602and60N. 

At this stage, the packets from each selected ones of the queues are 
routed to an assembly unit. Many methods for routing packets are known in the 
art. each can be used for routing packets from the queues, according to a given 
priority scheme, such as random eariy detection (RED), weighted random early 
detection (WRED), and the like. 
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According to the present invention, the queues are managed according 
to a priority scheme, in which the base layer queue 60o is assigned the highest 
priority Po and consecutive queues 60i. 6O2 and 60n are assigned respective 
descending priorities Pi, P2 and Pn- 

A packet within a queue, which was not transmitted within a 
predetermined period of time, is discarded and hence deleted, making room for 
new packets at the respective layer. This can be achieved in a plurality of ways. 
For example, each packet is assigned an expiration time stamp, before which the 
packet is considered valid for transmission and after which the packet can be 
discarded and deleted. The transmission router 74 collects packets from the 
queues 60, according to the priority scheme and transmits them over a variable 
bandwidth channel. 

Reference is further made to Figure 5, which is a schematic illustration 
of an assembly mechanism structure, constructed and operative in accordance 
with another preferred embodiment of the present invention. The structure 
includes a reception router 76, a layer assembler 78 and a plurality of queues, . 
generally referenced 80. 

The reception router 76 receives a plurality of packets from the variable 
bandwidth and distributes them according to the respective layers, among 
queues 8O0, 8O1, 8O2 and 80n. Base layer packets are directed to queue 8O0, 
Layer 1 A packets are directed to queue 8O1, Layer 2 A packets are directed to 
queue 8O2 and Layer N A packets are directed to queue 80n. 

As presented in the example of Figure 5, not all of the packets in each 
of the layers have arrived at a selected point in time, at the receiving end. With 
respect to the base layer, all of the frames have arrived. With respect to layer 1 , 
the packets respective of frame O, frame 1 and frame M have arrived and the 
packet respective of frame 2 has not. With respect to layers 2 and N, the packets 
respective of frame 0 and frame M have arrived and the packets respective of 
frames 1 and 2 have not. 

Accordingly, the output pictures for frames 0 and M can be provided at 
the highest quality, the output picture for frame 1 can be provided at the second 
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lowest quality (layer 1 ) and the output picture for frame 2 can only be provided at 
the lowest quality (layer 0). 

Layer assembler 78, retrieves the frame representations from all of the 
queues 80 and produces a media stream at the output thereof. 

Reference is now made to Figure 6, which is a schematic illustration of 
an apparatus, generally referenced 100, constructed and operative in accordance 
with another prefen-ed embodiment of the present invention. Figure 6 provides an 
overview of an entire apparatus for transmitting and receiving a media stream 
over a variable bandwidth channel. 

Apparatus 100 includes a transmission system 102 and a reception 
system 104. Transmission system 102 includes a stream analyzer 110, a layer 
structure storage unit 112, a layering generator 114, a layer storage unit 116, a 
packetizing unit 118 and a transmission controller 120. Layer structure storage 
unit 112 is connected to stream analyzer 110 and to a layering generator 114. 
Layer storage unit 116 is connected between layering generator 114 and 
packetizing unit 118. Packetizing unit 118 is further connected to transmission 
controller 120. Stream analyzer 110 and layering generator 114 are both 
connected to a stream source. 

Reception system 104 includes a reception controller 130, a storage 
unit 132 and a layer assembler 104. Storage unit is connected to reception 
controller 130 and to layer assembler 104. Reception controller Is connected to 
transmission controller via a variable bandwidth channel, referenced 140. 

It is noted that transmission system 102 and reception system 104 can 
be located at two ends of a communication channel of any given distance or co- 
exist In the same machine. 

Stream analyzer 110 receives a media stream, which can include a 
video stream, an audio stream, a media command stream or a data stream. 
Stream analyzer 110 processes the stream and detennines a set of compression 
command, which are directed at multi-layer compression of the received media 
stream, in accordance with a plurality of requirements such as minimal quality 
limit, minimal bandwidth limit, and the like. 
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The set of compression command is then stored in layer structure 
storage unit 112. Layering generator 114 retrieves the set of commands from 
storage unit 112 and compresses the incoming media stream, according to the 
quantizing scale values set forth by stream analyzer 110. It is noted that the 

5 Incoming media stream can either be non-compressed or compressed media 
stream (such as an l\/IPEG compressed stream). Layering generator 114 stores 
the produced layers in storage unit 116. 

Packetizing unit 118 retrieves portions of the layers stored in storage 
unit 116 and produces packets which are then transmitted by transmission 

10 controller 120 to reception controller 130, via over channel 40. Reception 
controller 130 stores the received packets in storage unit 132. Layer assembler 
134 retrieves frame data, according to layers, from the packets in storage unit 
132 and reassembles a valid media stream therefrom. It is noted that this valid 
media stream is at most at the quality of the incoming media stream, provided to 

15 stream analyzer 1 1 0. 

Reference is further made to Figure 7, which is an illustration of a 
method for operating the transmission system of the apparatus of Figure 6, 
operative in accordance with another preferred embodiment of the present 
invention. It is noted that the steps as listed in conjunction with this method are 

20 described for a single media stream. Nonetheless, as will be described further 
below, these steps apply for a structure, which processes a plurality of media 
streams and combines them for joined transmission over a single communication 
line. 

In step 150, a media stream is received. With reference to Figure 6, 
25 stream analyzer 110 receives a media stream. The media stream is provided 
either in a full or compressed format. In case where the received stream is 
provided in an analog format or in a digitized non-compressed format, then, 
preliminary digitization and compression procedures can be applied thereon. 

The received media stream is processed (step 152) and a plurality of 
30 quantizing values, respective of quality values, are determined (step 154). With 
reference to Figure 6, stream analyzer 110 processes the media stream and 
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determines preliminary parameters, which will be later used in the layering 
procedure. These values can be stored in storage unit 112 or used immediately 
(step 156). 

In step 158. a plurality of derivative media streams are determined. 
5 each at a different quality, according to a respective quantizing value. It is noted 
that the quantizing value can be detemilned for each frame or even each macro- 
block. With reference to Figure 6. layering generator produces a plurality of 
streams, each according to the previously detemilned quantizing values (Figures 
2A, 2B, 2C. 2D and 2E). 
^° step 160, the lowest quality derivative stream is determined as a 

base media stream. With reference to Figure 6. the layering generator 114 
detemriines the highest quantizing value stream as a base stream layer (Figure 
2Aand3A). 

In step 162, a delta (A) stream is detemilned between each pair of pair 
of successive derivative media streams. With reference to Figure 6, the layering 
generator determines a A stream layer for each pair of successive derivative 
media streams (Figures 3B, 3C. 3D and 3E). The base layer and the delta layers 
can further undergo a packetizing procedure, with respect to predetermined 
communication or transport protocols, by packetizing unit 118 (see also Figure 5). 
^° step 164, the base layer media stream and the respective delta 

layers are transmitted over a communication channel, according to a 
predetermined priority scheme. With reference to Figure 6. the transmission 
controller transmits the base and delta layers over the variable bandwidth 
transmission line 140. 

25 Reference is further made to Figure 8, which Is an illustration of a 

method for operating the reception system of the apparatus of Figure 6. operative 
in accordance with another prefen-ed embodiment of the present invention. 

In step 170. a base media stream is received. It is noted that the 
compression of the base media stream is directed tc comply at all times with the 

30 minimal bandwidth, which is available over the communication line. With 
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reference to Figure 6, the reception controller 130 receives a plurality of packets, 
which comprise the base layer respective of the lowest quality media stream. 

In step 172, the reception of at least one derivative media stream is 
detected. With reference to Figure 6, the reception controller 130 detects the 
5 presence of incoming packets, respective of layers, higher than the base layer. It 
is noted that the base layer packets as well as any received delta layer packets 
can be stored in storage unit 132. 

In step 174, the base media layer frames are combined with respective 
and consecutive delta layer frames, so as to form a valid media stream. With 
10 reference to Figure 6, the layers assembler 134 retrieves packets from storage 
unit 132 and reconstructs a valid media stream therefrom. 

In step 178, excess derivative media information Is deleted. With 
respect to Figure 6, the reception controller deletes received packets which were 
not used during media stream reconstruction and the transmission controller 120 
15 discards packets, which were not transmitted within a predetermined time period. 

It Is noted that the above layering scheme can be applied to any bjt- 
costly aspect, such as Include or exclude fields, temporal resolution, spatial 
resolution, luminance or chrominance, and a combination thereof. 

In accordance with another aspect of the present invention, there Is 
20 provided a method, which manages the transmission of a plurality of media 
streams over a single communication line. 

Reference is to Figures 9A. 9B, 9C and 9D, each providing a schematic 
Illustration of the layering distribution of a different media stream on the same set 
of layers, at different quantizing values. With reference to Figure 9A, the streams 
25 to be processed include M media streams, referenced 200 (media stream A), 202 
(media stream B), 204 (media stream C) and 206 (media stream M). The layer 
structure Includes N layers, arranged in N queues referenced 210 (layer 0 
queue), 212 (layer 1 queue), 214 (layer 2 queue) and, 216 (layer N queue). 

Media stream A 200 is processed according to the following quantizing 
30 values Q=4 (base layer - reference 210), Q=3 (first delta layer - reference 212), 
Q=2 (second delta layer - reference 214) and Q=1 (last delta layer - reference 
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216). which ultimately yields the original, highest quality media stream, as 
received before processing. 

With reference to Figure 9B, media stream B 202 is processed 
according to the following quantizing values Q=20 (base layer - reference 210). 
5 Q=1 0 (first delta layer - reference 212), Q=5 (second delta layer - reference 214) 
and Q=1 (last delta layer - reference 216). 

It is noted that, statistically, the arrangement as presented in Figures 
9A and 98 provides higher quality priority to media stream A over media stream 
B, which at the base layer is compressed according to a quantizing value of Q=5. 
10 guaranteed to be received at the receiving end. where media stream B is 
compressed at a quantizing value of Q=20 at the base layer and only the second 
delta layer represents a compression at a quantizing value of Q=5. This inherent 
characteristic of the present invention provides a user with control over the 
priority, regarding quality, which can be set for each of the media streams, before 
15 transmission. 

With reference to Figure 9C, media stream C 204 is processed 
according to the following quantizing values Q=10. Q=8, Q=6 and Q=1. 
Comparing with the quantizing schemes of Figures 9A and 9B. the "upgrading" 
scheme of each of the streams, can also be prioritized, where the first delta layer 
20 of one stream is quantized at a higher level than that of another stream, thus 
ensuring that given a selected bandwidth, one stream will be reconstructed at a 
higher quality than the other. 

With reference to Figure 9D, media stream D 206 is processed 
according to the following quantizing values Q=6, Q=5 Q=4 and Q=2. This layer 
25 quantizing scheme assumes that a bandwidth, suitable for Q=1 will most probably 
not be available, but a bandwidth for Q=2 will be. In that case, the highest delta 
layer is compressed at Q=2 rather than Q=1 . 

Reference is now made to Figure 10, which is a schematic illustration of 
an apparatus, generally referenced 300 for down streaming media through a 
30 plurality of communication channels, constructed and operative in accordance 
with another prefen-ed embodiment of the present invention. System 300 includes 
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a plurality of transmission systems 302, 304, 306 and 316 and a plurality of 
reception systems 310, 312, 314, 318 and 320. Each of the transmission system 
302, 304 and 306 processes a plurality of media streams at a plurality of 
quantization levels and transmits them over a high capacity channel, generally 

5 referenced 308, to reception systems 310, 312 and 314. It Is noted that each of 
the transmissions of each of the transmission systems 302, 304 and 306 can be 
received at any of the reception systems 31 0, 312 and 314. 

Each of the reception systems 310, 312 and 314 can provide the 
received data in a plurality of formats such as pass through, referenced 322 (e.g., 

10 the data further sent as It was received), MPEG video program stream, 
referenced 324 or a converted MPEG transport stream, referenced 326. 

It Is noted that the received data can further be processed Into new 
layers and transmitted further to a new location over a multi-channel network. An 
example is set by transmission system 316, which receives an assembled media 

15 stream from reception system 314. Transmission system 316 processes the 
received media stream into a plurality of layers and further transmits them to 
reception systems 318 and 320. It is noted that the quality of the result of this 
further transmission can not exceed the quality of the streams as received by 
reception system 314. 

20 Reference is now made to Figure 1 1 , which is a schematic illustration of 

a video re-multiplexing system, generally referenced 350, constructed and 
operative in accordance with another preferred embodiment of the present 
invention. 

System 350 includes a plurality of stream processors 352A, 352B, 
25 352C and 352N, a plurality of queue units 358A, 358B, 358C, 358D and 358N, a 
router 360, a control unit 362, a layer assembler 364, a packetizing unit 366 and 
a media decoder array 368. 

Each of stream processors 352A, 352B, 352C and 352N includes a 
stream analyzer unit, generally referenced 354 and a layering generator, 
30 generally referenced 356, respectively connected there between. 
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Layering generators 352A. 352B. 352C and 352N are all connected to 
queues 358A. 358B. 358C, 358D and 358N. It is noted that each of the layering 
generators can produce layers at a number, which does not exceed the number 
of the queues. 

5 Queues 358A. 358B. 358C, 358D and 358N are all connected to router 

360. It is noted that router 360 can be replaced with a respective switch, for 
enhanced speed. Router 360 is further connected to control unit 362 and to layer 
assembler 364. Layer assembler 364 is further connected to media decoder array 
368 and to packetizing unit 366. 

10 Each of stream analyzers 358A. 3588, 358C and 358M receives a 

media stream from a respective stream source. Each stream analyzer analyzes 
the received media stream and determines how and to what extent should that 
stream be layered. The decision can be perfonned according to a plurality of 
parameters such as the complexity of the information embedded in the stream, 

15 the priority which this stream has over the rest of the media streams, the 
allocated media stream bandwidth, the allocated overall communication channel 
bandwidth, and the like. 

The layering generator 352. connected to each of the stream analyzers 
354 performs the actual layering process and determination of the base layer and 

20 the associated differential (delta) layers. Each layering generator 356 then 
distributes all of the generated layers between the queues 358A. 358B, 358C, 
358D and 358N, where the base layer from each layering generator is stored in 
queue 358A, the first delta layer is stored in queue 358B and so on. Router 360 
retrieves layered media data from each of the queues 358A, 358B, 358C, 358D 
25 and 358N, according to a scheme multiplexing scheme received from the control 
unit 362. 

Layer assembler 364 reconstructs each of the streams to the extent 
available with respect to the retrieved layers. It is noted that the content of some 
queues, of higher order (higher layers) can be discarded, in accordance with a 
30 plurality of parameters such as time limit, volume limit, importance criterions and 
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the like. Layer assembler 364 provides the reassembled streams in an encoded 
format as output. 

Layer assembler 364 further provides the reassembled streams to 
packetizing unit 366 and to media decoder an-ay 368. Packetlzing unit 366 

5 encapsulates portions of the reassembled streams, according to any desired 
communication protocol such as Ethernet, ATM, TCP/IP, MPEG-Transport and 
the like. Media decoder array includes a plurality of decoding units (not shown) 
each decoding a selected one of the streams and providing it as output In the 
form of a media stream. Accordingly, the system 350 operates a stream 

10 multiplexer, which is able to multiplex a plurality into a given bandwidth. CBR or 
VBR, without wasting any bandwidth volume. 

It is noted that the method and apparatus according to the present 
invention can be implemented either in hardware, in software or in a combination 
thereof. 

15 It will be appreciated by persons skilled in the art that the present 

invention is not limited to what has been particularly shown and described 
hereinabove. Rather the scope of the present invention is defined only by the 
claims, which follow. 
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CLAIMS 

Method for providing scalable bandwidth transmission of a media stream, 
the rnethod comprising the steps of: 

determining a plurality of compression levels for said media stream; 

and 

defining the highest level of compression of said compression levels as 
a base level. 

The method according to claim 1 , further comprising the steps of: 

compressing said media stream according to said compression levels, 
thereby producing a plurality of compressed versions of said media stream; 

and / 

defining a selected one of said compressed versions a base layer, said 
selected compressed version being compressed according to said base 
level. 

The method according to claim 2, further comprising the step of producing a 
supplemental layer for each successive pair of said compressed versions. 

The method according to claim 2, further comprising the step of providing 
said base layer as a reconstructed media stream. 

The method according to claim 3, further comprising the step of producing a 
reconstructed media stream from said base layer and at least a portion of at 
least one of said supplemental layers. 

The method according to claim 2. further comprising the step of transmitting 
said base layer to a remote location. 

The method according to claim 3, further comprising the step of transmitting 
said base layer and at least a portion of at least one of said supplemental 
layers, to a remote location. 

The method according to claim 5, further comprising the step of transmitting 
said reconstructed media stream to a remote location. 
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1 9. The method according to claim 5, further comprising the step of packetizing 

2 said reconstructed media stream. 

1 10. The method according to claim 9, further comprising the step of transmitting 

2 said packetized reconstructed media stream to a remote location. 

1 11. The method according to claim 1 , wherein said media stream comprises one 

2 of the list consisting of: 

3 MPEG encoded media stream; 

4 Original media stream; 

5 JPEG media stream; 

6 Video stream; 

7 Audio stream; 

8 Data stream; 

g A stream containing a plurality of streams. 

1 12. Method for reconstructing a portion of a media stream from a portion of a 

2 base layer stream and a respective portion of at least one supplemental 

3 layer stream comprising the step of adding said respective portion of at least 

4 one supplemental layer stream to said base layer stream. 

1 13. Method for providing scalable bandwidth transmission of a plurality of media 

2 streams, the method comprising the steps of: 

3 determining a group of compression levels for each said media 

4 streams; and 

5 defining the highest level of compression of said compression levels of 

6 each said media streams, as a base level thereof. 

1 14. The method according to claim 1 3, further comprising the steps of: 

2 compressing each said media streams according to a respective one of 

3 said groups said compression levels, thereby producing a group of 

4 compressed versions, for each said media streams; and 

5 defining a selected one of said compressed versions of a selected 

6 group of compressed versions, a base layer, said selected compressed 
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7 version being compressed according to said base level, of said selected 

8 group of compression levels. 

1 15. The method according to claim 14, further comprising the step of producing 

2 a supplemental layer for each successive pair of said compressed versions 

3 of a selected group, thereby defining a group of supplemental layers. 

1 16. The method according to claim 14, further comprising the step of associating 

2 between portions of different ones of said base layers. 

1 1 7. The method according to claim 1 5, further comprising the steps of: 

2 associating between portions of different ones of said base layers; and 

3 associating between portions of different ones of said supplemental 

4 layers of the same order, of different ones of said groups. 

1 18. The method according to claim 16, further comprising the step of providing 

2 said portions of said associated base layers as reconstructed media 

3 streams. 

1 19. The method according to claim 17, further comprising the step of producing 

2 reconstructed media streams, each from a respective one of said portions of 

3 said associated base layers and from at least one of a respective one of said 

4 portions of said supplemental layers. 

1 20. The method according to claim 16, further comprising the step of 

2 transmitting said associated portions of said base layers to a remote 

3 location. 

1 21. The method according to claim 17, further comprising the step of 

2 transmitting said associated portions of said base layers and at least 

3 selected ones of said associated portions of said supplemental layers, to a 

4 remote location. 

1 22. The method according to claim 19, further comprising the step of 

2 transmitting said reconstructed media streams to a remote location. 

1 23. The method according to claim 1 9, further comprising the step of packetizing 

2 said reconstructed media streams. 
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1 24. The method according to claim 23, further comprising the step of 

2 transmitting said packetized reconstructed media streams to a remote 

3 location. 

1 25. The method according to claim 13, wherein said media stream comprises 

2 one of the list consisting of: 

3 MPEG encoded media stream; 

4 Original media stream; 

5 JPEG media stream; 

6 Video stream; 

7 Audio stream; 

8 Data stream; 

g A stream containing a plurality of streams. 

1 26. Method for reconstructing portions of media streams from portions of a base 

2 layer streams and a respective portions of at least one supplemental layer 

3 streams, comprising the steps of: 

4 detecting an association between each said base layer streams and at 

5 least a respective portion of at least one respective supplemental layer 

6 streams; 

7 adding each said base layer streams and said associated at least a 

8 respective portion of at least one respective supplemental layer. 

1 27. Scalable bandwidth transmission media stream generating apparatus 

2 comprising: 

3 a media analyzer; and 

4 a layering generator connected to said media analyzer, 

5 said media analyzer receiving a media stream and determining a 

6 layering structure therefore, 

7 said layering generator processing said media stream according to said 

8 layering structure, thereby producing a base layer and a plurality of 
g supplemental layers. 
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1 28. The scalable bandwidth transmission media stream generating apparatus 

2 according to claim 27, further comprising a storage unit, connected to said 

3 layering generator, 

4 wherein said storage unit is divided to a plurality of queues, each said 

5 queues storing a selected one of said base layers and said supplemental 

6 layers. 

1 29. The scalable bandwidth transmission media stream generating apparatus 

2 according to claim 28, further comprising a routing controller, connected to 

3 said a storage unit, for retrieving selected successive ones of said base 

4 layers and said supplemental layers. 

1 30. The scalable bandwidth transmission media stream generating apparatus 

2 according to claim 29, further comprising a layer assembler, connected to 

3 said routing controller, for producing a reconstructed media stream from said 

4 base layer and at least a successive one of said supplemental layers, 

5 received from said routing unit. 

1 31. Scalable bandwidth transmission media stream generating apparatus 

2 comprising: 

3 a plurality of media analyzers; 

4 a plurality of layering generators, each associated with and connected 

5 to a selected one of said media analyzers, 

6 each said media analyzers receiving a different media stream and 

7 determining a layering structure therefore, 

8 each said layering generators processing said media stream according 

9 to said layering structure, thereby producing a base layer and a plurality of 

10 supplemental layers. 

1 32. The scalable bandwidth transmission media stream generating apparatus 

2 according to claim 31 , further comprising a storage unit, connected to said 

3 layering generator, 
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4 wherein said storage unit is divided to a plurality of queues, each said 

5 queues storing a selected one of said base layers and said supplemental 

6 layers. 

1 33. The scalable bandwidth transmission media stream generating apparatus 

2 according to claim 32, further comprising a routing controller, connected to 

3 said a storage unit, for retrieving selected successive ones of said base 

4 layers and said supplemental layers. 

1 34. The scalable bandwidth transmission media stream generating apparatus 

2 according to claim 33, further comprising a layer assembler array, connected 

3 to said routing controller, for producing a reconstructed media streams from 

4 said base layers and at least a successive ones of said supplemental layers, 

5 received from said routing unit. 

1 35. Method for providing scalable bandwidth transmission of a plurality of media 

2 streams, via a fixed bandwidth communication channel, the method 

3 comprising the steps of: 



4 determining a group of compression levels for each said media 

5 streams; 

6 defining the highest level of compression of said compression levels of 

7 each said media streams, as a base level thereof, wherein the total sum of 

8 the compressed versions of said media streams according to the respective 
g ones of said base levels, at any point in time, is less than said fixed 



10 bandwidth. 

1 36. The method according to claim 35, further comprising the steps of: 

2 compressing each said media streams according to a respective one of 

3 said groups said compression levels, thereby producing a group of 

4 compressed versions, for each said media streams; and 

5 defining a selected one of said compressed versions of a selected 

6 group of compressed versions, a base layer, said selected compressed 

7 version being compressed according to said base level, of said selected 

8 group of compression levels. 
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37. The method according to claim 36, further comprising the step of producing 
a supplemental layer for each successive pair of said compressed versions 
of a selected group, thereby defining a group of supplemental layers. 

38. The method according to claim 36, further comprising the step of associating 
between portions of different ones of said base layers. 

39. The method according to claim 37, further comprising the steps of: 

associating between portions of different ones of said base layers; and 
associating between portions of different ones of said supplemental 
layers of the same order, of different ones of said groups. 

40. The method according to claim 38. further comprising the step of providing 
said portions of said associated base layers as reconstructed media 
streams, where the total sum of bandwidths of each said reconstructed 
media streams does not exceed said fixed bandwidth. 

41 . The method according to claim 39, further compiising the step of producing 
reconstructed media streams, each from a respective one of said portions of 
said associated base layers and from at least one of a respective one of said 
portions of said supplemental layers, wherein the total sum of bandwidths of 
each said reconstructed media streams does not exceed said fixed 
bandwidth. 

42. The method according to claim 38, further comprising the step of 
transmitting said associated portions of said base layers to a remote 
location, wherein the total sum of bandwidths of said associated portions of 
said base layers does not exceed said fixed bandwidth. 

43. The method according to claim 39, further comprising the step of 
transmitting said associated portions of said base layers and at least 
selected ones of said associated portions of said supplemental layers, to a 
remote location, wherein the total sum of bandwidths of said associated 
portions of said base layers and at least selected ones of said associated 
portions of said supplemental layers does not exceed said fixed bandwidth. 
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44. The method according to claim 41 , further comprising the step of 
transmitting said reconstructed media streams to a remote location, wherein 
the total sum of bandwidths of each said reconstructed media streams does 
not exceed said fixed bandwidth. 

45. The method according to claim 41 , further comprising the step of packetlzing 
said reconstructed media streams. 

46. The method according to claim 45, further comprising the step of 
transmitting said packetized reconstructed media streams to a remote 
location wherein the total sum of bandwidths of each said reconstructed 
media streams does not exceed said fixed bandwidth. 

47. The method according to claim 35, wherein said media stream comprises 
one of the list consisting of: 

MPEG encoded media stream; 
Original media stream; 
JPEG media stream; 
Video stream; 
Audio stream; 
Data stream; 

A stream containing a plurality of streams. 

48. A method according to any of the claims 1-11 substantially as described 
herein above. 

49. A method according to any of the claims 1-11 substantially as illustrated in 
any of the drawings 

50. A method according to claim 12 as described herein above. 

51. A method according to claim 12 as illustrated in any of the drawings. 

52. A method according to any of the claims 1 3-25 substantially as described 
herein above. 
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1 53. A method according to any of the claims 13-25 substantially as illustrated in 

2 any of the drawings. 

1 54. A method according to claim 26 as described herein above. 

1 55. A method according to claim 26 as illustrated in any of the drawings. 

1 56. Apparatus according to any of the claims 27-30 substantially as described 

2 herein above. 

1 57. Apparatus according to any of the claims 27-30 substantially as illustrated in 

2 any of the drawings. 

1 58. Apparatus according to any of the claims 31-34 substantially as described 

2 herein above. 

1 59. Apparatus according to any of the claims 31-34 substantially as illustrated in 

2 any of the drawings. 

1 60. A method according to any of the claims 35-47 substantially as described 

2 herein above. 

1 61 . A method according to any of the claims 35-47 substantially as illustrated in 

2 any of the drawings. 
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